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Aims. We emphasize that the mention of the significance level when rejecting the null hypothesis (Ho which assumes that what is observed is 
pure noise) can mislead one to think that the H hypothesis is unlikely to occur with that significance level. We show that the significance level 
has nothing to do with the posterior probability of Ho given the observed data set, and that this posterior probability is very much higher than 
what the significance level naively provides. 

Methods. We use Bayes theorem for deriving the posterior probability of H being true assuming an alternative hypothesis H[ that assumes 
that a mode is present, taking some prior for the mode height, for the mode amplitude and linewidth. 
Results. We report the posterior probability of Ho for the p modes detected on HD49933 by CoRoT. 

Conclusions. We conclude that the posterior probability of Ho provide a much more conservative quantification of the mode detection than the 
significance level. This framework can be applied to any stellar power spectra similar to those obtained for asteroseismology. 
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1. Introduction 

In the fi eld of helioseismology, the null hypothesis Ho has been 
used by I Appourchaux et al. (2000J) to infer upper limits on the 
amplitude of g modes. The Ho hypothesis assumes that what 
is observed is pure noise. This hypothesis has been used for 
classical variab le stars for detecting peaks in a power spectrum 
dScargldll982l) . In these papers, the authors accepted the Ho 
and set an upper limit corresponding to a threshold level of say 
10%. Here we argue, that this threshold has been somewhat 
arbitrarily chosen a priori. By accepting or rejecting the Ho 
hypothesis, there is no discussion as to why borderline case 
should be rejected or accepted. This abrupt truncation between 
the good and the bad leads to a decision that could have been 
different if the threshold had been different. 

In the following sections, we first lay down the foundation 
for understanding the meaning of the Ho hypothesis, and then 
explain what is commonly misunderstood about the Ho hypoth- 
esis. We then derive, for specific cases encountered in helio- 
and asteroseismology, the posterior probability of Hq. We show 



* The CoRoT space mission, launched on 2006 December 27, was 
developed and is operated by the CNES, with participation of the 
Science Programs of ESA, ESA's RSSD, Austria, Belgium, Brazil, 
Germany and Spain. 



how one can use the formalism for real data such as that gath- 
ered by the CoRoT mission for HD49933 and then conclude. 

2. Significance level and the H hypothesis 

Fished dl925h devised the well known Fisher test for testing the 
null hypothesis (Ho). For that test, a threshold of 5% is com- 
monly used; the p-value quoted is the value of the test if it 
is less than this threshold. So, for instance, a result of 4.9% 
would result in rejecting the Ho hypothesis, while 5.1% would 
result in accepting the Ho hypothesis. When the Ho hypothe- 
sis is rejected, the reported p-value is used as a significance for 
the validity of not accepting the Ho hypothesis. The so-called 
borderline cases led in the medical field to findings related to 
effectiveness of medicine that were, sometimes, not proven by 
subsequent studies. The controversy about the use of p-values 
that occurred in the medical world is directly related to the 
abrupt and arbitrary cut-off of the threshold applied (be 5% or 
10%). Although the relevance of what has been found in the 
medical field could seem remote to most astrophysicists, it is 
indeed extremely relevant to understand that the improper use 
of p-values is the same as the improper use of the so-called 
significance level. 

Reporting a small number for the significance level should 
not be used for claiming the proper rejection of the Ho hypothe- 
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sis. The mistake is to ascribe a significance level to a measure- 
ment carried out only once, not repeated and spanning just a 
very small volume of the space of the parameters. When mak- 
ing an observation of random variable x, one wants to check 
the probability that what is observed could be due to noise. For 
that purpose, a test statistic is derived T(x). If one observes a 
value of x — X that would not reject the Ho, then one compute, 
p, the significance level orp-value, defined as: 



p = Po(T(x) > T(X)) 



(1) 



where Pq is the probability of having T(x) > T(X) when Ho 
is true. The test set by Eq. (1) is about checking that the sta- 
tistical test T(x) is compared with the value given by T(X). 
When computing the statistical test T{x), one does not span 
the space of X: one has x — X: this is the so-called point 
null hypothesis. In other terms, what are the probabilities that 
one has exactly that value of xl This is a completely different 
question from knowing how true Ho is, i.e. obtaining p(Ho\x). 
Bereer & Sellkd (11987b provided a way of deriving p(Ho\x) 
with respect to p(x\Ho) and p(x\Hi), where Hi is the alternative 
hypothesis (i.e. there is a signal). They obtained using Bayes 
theorem, the so called posterior probability of Ho given the 
observed data x: 



pWo\x) = 



p(u )p(x\n ) 



p(HoM*|H )+.p(H 1 )p(*|Hi) 



(2) 



where p(H ) and p(H\) are the probabilities ascribed to the 
H and Hi hypothesis, respectively. Note that what we really 
want to know is /?(H |x), the probability of H being true 
given the data we have, not the probability of the data x 
given Ho, i.e. p(x|Ho). In order to minimize the impact of the 
probabilities assigned to both hypothesis, we assume that 
these are equally probable (p Q =0.5): 



p(n Q \x) = l 



p(x\Hi) 

p(x\h ) 



(3) 



Under these assumptions ISellke et al. ( 2001 ) found that the 
probability p(Ho\x) of having Ho being true given some ob- 
served data x of a random variable X has a lower bound: 



p(Bo\x) > 1 - 



1 
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An immediate consequence is that for a significance level of 
1%, the odds against Ho are at least 10 to 1, and for 10%, the 
odds against Ho are at least 2.6 to 1 . In both case, the likelihood 
of wrongly rejecting Ho is much higher than what the p- value 
provides, by at most a factor 10 and 4 respectively. 



Sellke et al. ( 2001 ) were able to set a lower value to p(H \x) 



for almost an arbitrary alternative Hi hypothesis. We show that 
there is indeed a lower bound when one wants to detect peaks 
in power spectra. Hereafter, we give several examples of how 
one can derive in practice the posterior probability of Ho. 



3. Posterior probability for peak detection 

3.1. Long lived modes 

3.1 .1 . Mode height known a priori 

Here we take the case of a power spectrum for which we seek a 
peak restricted to a single frequency bin. The power spectrum 
has ax 2 with 2 d.o.f. statistics, for which a bin has reached a 
value x. We want to test if this could be the result of a true sine 
wave or due to noise. We have for the Ho hypothesis: 



p(je|Ho) = e-" v 



(5) 



The noise is assumed to be 1. For the alternative hypothesis 
Hi, we assume that there is a signal of a long lived mode, i.e. 
restricted to one bin, for which the mode height H is known 
and the mode is stochastically excited (like a stellar p mode). 
Then we have: 



p(jr|Hi) 



1 



,-xKi+m 



1 + H 



Equation (3) is then rewritten for our problem as: 



p(H \x) = 11 + 



1 



1 +H 



,xH/(l+H) 



Since the significance p = p(x\Ho), we finally have: 

-l 



p(H \x) = 1 + 



1 



1 +H' 



-H/(\+H) 



(6) 



(7) 



(8) 



It can be shown that the minimum of p(Ho\x) is reached for 
H = - lnp - 1, and with the value: 

^ (How =( i - e ^r (9) 

In that case the lower bound set by Eq. (4) is reached. 
3.1.2. Mode height unknown 

Of course, most of the time one does not know the height of 
the mode to be detected. We can assume a prior for the mode 
height (uniform distribution, gaussian, etc.). For example if 
we assume that the mode height is uniformly distributed over 
some range [0,// u ], p(x\Hi) is rewritten as: 



Hu Jo i + H 



(4) Then Eq. (3) can be rewritten as: 



(10) 



(11) 



It can be shown that there is a minimum reached at a value of 
H min by solving: 

— P"° -±—p-w+ H ')w = —5 — p-*w(i+«-J (12 ) 

#min Jo 1 + H' 1 + H m { n 



The minimum is then given by: 



jC(HoW= 1 + 



1 



l+H„ 



(13) 



T. Appourchaux, R. Samadi, M.-A. Dupret: On posterior probability and significance level 3 

Significance level = 0.100 Significance level = 0.100 



1.0 [ 



0.1 



4 6 
Mode height 



Significance level = 0.010 



10 




4 6 
Mode height 

Fig. 1. Posterior probability as a function of the known mode 
height (dashed line), or as a function of the mode height range 
(H u ) of the uniform prior (continuous line) for a significance 
level of 10%. For the known mode height, the minimum is 
reached for an height of 1.3 with a value of 38%. For the un- 
known mode height, the minimum is reached at a value of 40%. 
Posterior probability as a function of the known height (dashed 
line), or as a function of the maximum mode height of the uni- 
form prior (continuous line) for a significance level of 1%. For 
the known mode height, the minimum is reached for an height 
of 3.6 with a value of 1 1%. For the unknown mode height, the 
minimum is reached at a value of 12.7%. 

From Eqs. (8) and (9), we can deduce that 



fC(Hol*)> 1 



1 



spin p 



(14) 



It means that when there is more uncertainty about the possible 
height of the mode, one is less likely to reject the Ho hypothe- 
sis. 

3.2. Short lived modes 



Appourchaux! d2004l) reported on how one can detect a mode 
having a lifetime shorter than the observation time. . He sug- 
gested smoothing the power spectrum in order to enhance the 
signal-to-noise ratio and provided the analytical expression for 
Hq related to the summation over n bins of a x 1 statistic with 
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Fig. 2. Posterior probability for a short lived mode when 
smoothing the power spectra over 10 bins corresponding to a 
window twice as large as the mode linewidth. The posterior 
probability is given as a function of the known mode height 
(dashed line), or as a function of the mode height range (H u ) 
of the uniform prior (continuous line) and a uniform prior for 
the linewidth, for a significance level of 10%. 



constant mean; 

2 



and for Hi related to the summation over n 
bins of ax z statistic having different means due to the presence 
of the mode profile. Therefore, when the spectrum is summed 
over n bins, we can derive for the Ho hypothesis : 



r(«) 



(15) 



where the mean of the power spectrum is 1, while n is the mean 
of the smoothed power spectrum (for simplicity, we assumed 
that S — 1), and y(n) is the Gamma function. The significance 
level x is then given by solving: 



— r«« 

r(«) Jx 



(16) 



Assuming, that the mode has a known ampli tude A and a known 
linewidth F, we can write using Eq. (8) of Appourch aux] d2004h 
the following approximation: 



y(v) 



(17) 



where A and v, given in Ap pourchaux 1 20041) are obtained by 
integrating symmetrically around the central frequency of 
the mode, thereby ensuring that the signal is maximum; A 
and v are functions of the mode height H(= A 2 IjiY) and T. 
If we do not know the mode amplitude and its linewidth, we 
can specify a prior for the amplitude and the linewidth which, 
for instance, can be done using a model of mode excitation. 
Here we assume that the mode amplitude A and the linewidth 
T are independent (this would not be the case of H and F). If 
we use uniform priors, we then have: 



i r^ u a v 
p(*ih!) = — - \ -r? 
A u r u Jo Jo y(v) 



V-'Wdr' 



(18) 



This equation would be similar if we were to have a uniform 
prior on the mode height, A would then be replaced by H. 
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Fig. 3. Lower bound to the posterior probability as function of 
the significance level for the known mode height (dashed line) 
and for the uniform prior (solid line). 



Combining Eqs. (16) and (17) in Eq. (3), we can then obtain 
the posterior probability p(Ho\x) when the mode amplitude and 
linewidth are known. If we replace Eq. (17) by Eq. (18), we 
then obtain the probability p(Ho\x) when the mode amplitude 
and linewidth are not known. 



3.3. Discussion 

Figures 1 shows the posterior probability for long lived modes 
for a known mode height (Eq. 8) and an unknown mode height 
(Eq. 11) for two different significance levels. Figure 2 shows 
the posterior probability for short lived modes for a known 
mode height and linewidth (Eqs. 15 and 17) and for an un- 
known mode height and linewidth (Eqs. 15 and 18, with H 
replacing A) for a single significance level; even a significance 
level of 1 % does not provide a better rejection of the Ho hypoth- 
esis. These posterior probabilities have a lower bound which 
means that even a very low significance level is no guarantee 
for positive detection! 

It is also counter intuitive that the posterior probability in- 
creases when the mode height (known or unknown) increases. 
We recall that the significance level p corresponds to the level 
x at which the peak has been observed (i.e. it is p = e~ x for 
long lived modes). If we assume a priori that the mode has a 
high mode height, then the observation at a low significance 
level indicates that our assumption on the high mode height is 
not correct, and that the data dismisses the a priori made on the 
mode height. In other words, it is more probable that the Ho 
hypothesis is true. 

In case of the absence of an alternative hypothesis, it is bet- 
ter to set a low significance value that will result in a likely re- 
jection of the Ho hypothesis. Figure 3 shows the lower bouncQ 
set by Eq. (4) compared to the minimum found using Eq. (11) 
for the uniform prior on the mode height . 



4. Application to the CoRoT data: HD49933 

The computation of the posterior probability has been applied 
to CoRoT for illustrative purposes. The da ta used are those of 



the fi rst initial run performed on HD49933 ( Ap pourchaux et al 



2008). The objective was to provide an objective way of detect- 
ing oscillation modes in HD49933 that could be applied to any 
other star. 

The methodology used for deriving the posterior probabil- 
ity is as follows: 

- we compute t he power spectrum from the detrended time 
series, as in Appourchau x et al. ( 2008 ). 

- we smooth the spectrum over n bins using a boxcar. 

- we select 30 50-//Hz wide windows starting at 1200 pHz 
(the 50-^Hz window corresponds roughly to half the large 
frequency separation). 

- For each window, we compute the median in the window of 
the smoothed spectrum (it provides an estimate of the mean 
noise level were the modes not present). 

- The smoothed spectrum is normalized in each window by 
dividing by the median and multiplying by the number of 
smoothing bins n (it provides values commensurate with 
these of Eq. 15). 

- we apply the Ho hypothesis using a detection probability of 
a signal being due to noise of 10% over all the 30 windows, 
taking into account the fact that in each window the num- 
ber of independent bins is 50 5v~ l rT l (Sv is the frequency 
resolution of the original power spectrum). The detection 
probability is then 0.1 (N w y l (506v~ 1 n~ l y l per indepen- 
dent bin. 

- we then solve Eq. (16) for Xdet given the detection level 
given above. 

- In each window, we then select the bins that are greater than 
Xdet, i-e. we accept or reject the Ho hypothesis. 

- After the selection, we keep the greatest value x max found 
in the window corresponding to the central frequency of 
the mode (See Eq. 17). 

- we then compute the posterior probability of Ho given by 
Eq (3) using Eqs. (15) and (18) assuming some prior on the 
mode height and linewidth as described below. 

- For comparison, we also compute the significance level as 
given by Eq. (16) from the value of x max . 

The theoretical a mplitudes for HD49933 are derived from 
Samadi et al.1 d2009ah using an adiabatic treatme nt of radiative 



The same lower bound as Eq. (9) for a known mode height 



transfer, and the excitation rate as calculated in ISamadi et al 
(2009b). The theoretical linewidths have been computed 
with the non-adiabatic pulsation code MAD. This code in- 
cludes a time-dependen t convection treatment described in 
Grigahcene et al.l d2005l) : it takes into account the role played 
by the variations of the convective flux, the turbulent pressure, 
and the dissipation rate of turbulent kinetic energy. This treat- 
ment is non-local, with three free parameters a, b, and c corre- 
sponding to the non-locality of the convective flux, the turbu- 
lent pressure and the entropy gradient. We take here the values 
a = 10, b = 3, and c = 3.5 obtained by fitting the convec- 
tive flux and turbulent pressure of 3D hydrodyn amic Simula- 
tions in the upper overshooting region of the Sun (iDupret et al 
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2006). According to lGrigahcene et a il j2005l) . we introduced a 
free complex parameter ft in the perturbation of the energy clo- 
sure equation. We use here the value /3 = -3i, which leads 
to a good agreement between the theoretical and observed 
linewidths and phase lags in the range of solar pressure modes. 
AID stellar model obtained with the code CESAM was used 
for our non-adiabatic computations. It has a solar metallicity 
and repr oduces the effective temperature and gravity of HD 
49933 ( Sam adi et al.1 12009a). The amplitudes and linewidths 
are shown on Figure 4. We have linearly interpolated both 
curves to provide a continuous prior with frequency. 

The uniform priors for amplitude and linewidth are derived 
from the theory by taking into account of an uncertainty factor 
in the theoretical model. For amplitude, we assumed that the 
maximum is V2 larger than given on Fig. 4 (twice in energy); 
for linewidth, we assumed that the maximum is twice greater 
than given on Fig. 4. Note that a larger prior increases the pos- 
terior probability, as shown on Figs. 1 and 2. We assumed that 
the noise floor in HD499 33 is given by the photon n oise, which 
is about 0. 15 ppm 2 /yuHz (lAppourchaux et al ] |2008h . 

Figure 5 shows the results of the procedure described 
above. It is clear that the posterior probability is higher than 
the significance level, i.e. the posterior probability provides 
a more conservative number (Hq more likely). The smooth - 
ing procedure also s hows two effects that were predicted by 
Appourc hauxl ( 20041) : first, short lived modes are easier to de- 
tect when the spectrum is smoothed, second, long lived modes 
are more difficult to detect when the spectrum is smoothed. 
The first effect manifests itself in the larger number of detected 
modes at higher frequency and by the decrease of the prior 
probability (i.e. the signal is more likely). The second effect is 
seen at low frequency where a couple of modes have their prior 
probability increased to non-negligible value after smoothing 
(i.e. the signal is less likely). When w e compare with modes 
reported by lAppourchaux et al.1 (2008). we find that more than 
85% of the I = - 2 mode pairs and / = 1 modes are recov- 
ered. An additional mode at 2579 fiHz is de tected that could be 



an / = 1 according to the identification of lAppourchaux et al 
(2008). 



5. Conclusion 

The significance level refers to the significance of the data 
given the hypothesis, while we are interested in the poste- 
rior probability of the null hypothesis given the data. Here we 
showed that for a significance level of 10%, the posterior prob- 
ability of the null hypothesis is at least 38% when there is no 
alternative hypothesis. We have shown how one can in prac- 
tice calculate and compute the posterior probability for the null 
hypothesis. This has been applied to several examples and to 
the CoRoT data. We have shown, for the first time, how one 
can assess the detectability of short lived p modes in a power 
spectrum. The methodology can be applied to any stellar power 
spectrum for which theoretical expectations are available. 

Acknowledgements. We are grateful to John Leibacher for checking 
the English and for correcting several typos. We are thankful to the 
referee for making the paper clearer. 
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Fig. 4. (Left) Theoretical mode linewidth of HD49933 as a 
function of frequency. (Right) Theoretical mode amplitude of 
HD49933 as a function of frequency. 



10° 
a) 10- 2 

U 

| 10~9 
M 10- 6 

O 

Jj io- 10 

in 
O 

1Q-14 




1200 1400 1600 1800 2000 2200 2400 2600 
frequency (in fMz) 




1200 1400 1600 1800 2000 2200 2400 2600 
frequency (in ^Hz) 

Fig. 5. (Top) Comparison of the posterior probability (solid 
line) to the significance level (dashed line) as a function of 
frequency for a spectrum smoothed over 5 bins (« 1//Hz). 
(Bottom) Posterior probability as a function of frequency for 
a spectrum smoothed over 5 bins (solid line) and over 50 bins 
(dashed line), respectively as lyuHz and a 10/iHz 
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